Parameters for Quantifying the Interaction with Spoken Dialogue Telephone Services

When humans interact with spoken dialogue systems, parameters can be logged which quantify the flow of the interaction, the behavior of the user and the system, and the performance of individual system modules during the interaction. Although such parameters are not directly linked to the quality perceived by the user, they provide useful information for system development, optimization, and maintenance. This paper presents a collection of such parameters which are now considered to be recommended by the International Telecommunication Union (ITU-T) for evaluating telephone-based spoken dialogue services. As an initial evaluation, a case study is described which shows that the parameters correlate only weakly with subjective judgments, but that they still may be used for predicting quality with PARADISE-style regression models.